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ABSTRACT 

The 1990s . brought a new notion of accountability for public 
colleges and universities. The goal shifted from accounting for expenditures 
to accounting for results. Academics, who were now expected to set goals and 
evaluate results while cutting costs, opposed the application of this model 
to academia. Public higher education had adopted Austin’s Resource and 
Reputation Model, which states that quality comes from the quantity of campus 
resources, the quality of admitted students, and the reputation of faculty 
research. The resistance to focusing on institutional results led to 
criticisms by business and government, and the states stepped in. The six 
regional, accrediting agencies made assessing student outcomes a requirement 
for accreditation. Performance reporting added meeting state needs to the 
earlier assessment practices. Reporting these results also had the tacit goal 
of increasing state funding, at least for campus leaders. By 2000, 30 states 
had performance reports; yet 49% of State Higher Education Finance Officers 
say that accountability reporting has had little or no effect on 
institutional performance. Nearly 90% of senior officers say they are 
familiar with' performance funding programs, while over 60% of chairs report 
little or no familiarity, which is a possible explanation for the lack of 
impact of the programs. (Contains 17 tables and 16 references.) (NB) 
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Introduction 

It takes courage to give a Keynote Address on accountability, 
reporting, and performance to institutional researchers from California’s 
community colleges. You have the unenviable task of collecting and 
analyzing the campus data mandated by state policies for performance 
reporting, budgeting, and funding. Your work does not make you popular 
on campus, for one of Burke’s Laws states that the interests of professors 
in assessing performance is in direct proportion to the distance from 
their campuses and their departments. My task today puts me in the 
plight of Niccolo Machiavelli on his deathbed. Priests surrounded the 
notorious author of The Prince urging him to recant and renounce his 
writings. With his last drying breath, Machiavelli muttered: “Now is not 
the time to make new enemies.” 

Like Machiavelli, I have written too much for too long on the 
necessity of reporting and rewarding the results of public colleges and 
universities to alter my views. It is hard say anything on this heated topic 
without alienating either campus or state leaders. It seems impossible to 
say anything sensible on this subject without alienating both groups. 
Despite these difficulties, I want to explore today a series of questions. 
What is the reason for the recent demands for accountability from public 
higher education? Why have performance programs become so popular 
in state capitals and so controversial on public campuses? How effective 
has performance reporting, budgeting, and funding been in improving 
campus performance. What changes could make performance policies 
more meaningful to professors? And why are these policies tailored to the 
mission of community colleges? 
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TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



The New Accountability 



The 1990s brought a new notion of accountability for public colleges 
and universities. The goal shifted from accounting for expenditures to 
accounting for results. Public higher education had become too 
important and too costly to states to ignore campus results. The shift 
gained momentum from the movements to reengineer business and 
reinvent government (Osborne 8s Gabler 1992; Hammer 8s Champy 
1993). They preached a novel gospel for business and a new heresy for 
government and higher education. Their creed proclaimed that all 
organizations not only could, but also must, improve quality while 
cutting costs. Institutions could demonstrate accountability while 
decentralizing authority by being tight on setting goals and assessing 
results but loose on the means of achieving their objectives. These 
theories suited the times. Success in the new information era demanded 
autonomy to encourage the creativity and ingenuity of knowledge 
workers, but it also required direction as well as decentralization. 
Managing, measuring, and rewarding results became the new trinity. 
Like all creeds, it proved easier to proclaim than to practice. 

The Campus Reaction 

Although academics conceived many of these management theories, a 
common reaction on campus called them all right for business, and 
maybe for government, but anathema for academe. Outsiders could have 
predicted that the accent on efficiency would arouse opposition on 
campus, but few would have guessed that the focus on quality would 
prove a greater obstacle. Colleges had declared “Quality Job One” 
centuries before Ford. Unfortunately, the academic community never 
determined nor defined with any precision the objectives of 
undergraduate education nor developed systematic methods for 
assessing campus performance. Quality in higher education is an elusive 
and subjective attribute that is seldom easy to evaluate objectively and 
always difficult to measure quantitatively. In addition, higher education 
is a complex enterprise with multiple goals, delivered by different types of 
institutions pursuing diverse missions and educating students with a 
broad range of natural abilities and academic preparations. 

By default and perhaps preference, the perception of institutional 
quality depends largely on what Alexander Astin (1985) labeled the 
Resource and Reputation Model. Quality comes from the quantity of 
campus resources, the quality of admitted students, and the reputation 
of faculty research. This Model -- based entirely on inputs of funding, 
students, and faculty -- says nothing about the quality or quantity of the 
services provided to students, states, and society. Quality products from 



colleges, like quality outputs from computers, depend mostly on good in, 
good out. 

Though the diversity of American higher education is a wonder of the 
world — with its marvelous mix of community and technical colleges, 
liberal arts and specialized campuses, and comprehensive and research 
universities — at home, the Research and Reputation model confers the 
hallmark of excellence only on large, graduate, research universities and 
small, selective, liberal arts colleges. This model condemns most colleges 
and universities, and especially community colleges, to perpetual 
inferiority. Student selectivity, big budgets, faculty research, and 
graduate education are the characteristics that count. 

External Criticism 

The resistance of higher education - especially research universities - 
to focus on institutional results led to mounting criticism in state 
capitals and the business community. Critics complained about the 
quality and quantity of faculty teaching and student learning, the 
preoccupation with graduate studies and research and neglect of 
undergraduate education, the encouragement of mission creep and 
program sprawl, and the bourgeoning of administrative positions and 
support staffs. They condemned higher education for its failure to 
constrain costs and tuition and for its reluctance to reform and 
restructure like business and industry. Although the criticism hit mostly 
at research universities, the fallout also covered community colleges. 

Antagonists accused colleges and universities of admitting too many unqualified 
students, of devoting too much time to correcting their deficiencies, of graduating too few 
of the students admitted, and of permitting too many to take too long to degrees. At the 
same time, they blamed public campuses for not educating more of the population. Most 
of all, critics claimed that increasing numbers of alumni lacked the knowledge and skills 
required for successful careers and meaningful lives in a knowledge and information 
society. After the move of health care to managed care, external critics frequently saw 
higher education as the last refuge of a provider-driven enterprise, designed more to 
satisfy the aspirations of administrators and faculty than the needs of students and 
society. Although friends of higher education quarreled with some of the particulars, 
many thought the academy guilty on most of the indictment (Wingspread Group 1993). 

The Assessment Movement 

As so often happens when the academic community seems reluctant 
to deal with perceived problems, the states step in. In the middle to late 
1980s, Governor’s, legislatures, and coordinating boards mandated 
assessment policies for public colleges and universities in two-thirds of 
the states. This first step seemed soft, for states responded to 
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institutional resistance by leaving the details to campus determination. 
The six regional accrediting agencies followed by making assessing 
student outcomes a requirement for accreditation (Nettles et al. 1997). 
Assessment asked colleges and universities to identify the knowledge and 
skills their graduates should possess, to develop methods for measuring 
achievement of these goals, and to use the results to improve 
institutional performance. It promoted a new notion of institutional 
quality based on results not resources, on performance not prestige. This 
concept of quality seemed tailored to the traditional mission of 
community colleges, which were created to meet the needs of their 
communities and their citizens. Assessment also appeared to promise a 
marriage between the unlikely mates of external accountability and 
institutional improvement. Improving student learning from entry to 
graduation signaled success in the new accountability for results and in 
responsiveness to societal needs (Burke 1999). 

Assessment spread swiftly across the county, but its impact on 
campus rarefy ran deep. By 1993 Campus Trends reported, “fully 97 
percent of institutions had some type of assessment activity...” (El- 
Khaws, 17). But 55 percent of the community colleges and 42 percent of 
public baccalaureate campuses admitted having only limited activities. A 
few colleges made assessment the centerpiece of their culture, but in 
most institutions it remained a cottage industry. Exciting practices 
flourished in a hundred places on campus, while the institution plodded 
it traditional path. Decentralization and autonomy on campus smile on 
individual innovations but stifle institutional reforms. Assessment had 
ardent advocates but little campus acceptance. All too often, a tacit 
bargain exists among professors. “You can do anything you like, just 
don’t ask me, my department, or my institution to do it” (Burke 1999). 
The failure of assessment to achieve accountability led states to adopt 
performance policies with teeth. 

Performance Reports 

If the assessment mandates represented decentralization with little 
direction, performance reporting - begun mostly in the 1990s — added 
direction as a price of decentralization. The purposes of performance 
reporting, like assessment, included improving institutional performance 
and demonstrating external accountability, but the new policy added 
meeting state needs, especially in economic development. Reporting 
results also had the tacit goal of increasing state funding, at least for 
campus leaders. In contrast to the assessment policies, performance 
reporting generally required comparability among campuses, although 
the reports urged comparisons only among the same types of 
institutions. The policies often required all public colleges or universities 
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- whatever their type or mission - to report on a common list of statewide 
indicators. 

A 1997 Survey from the State Higher Education Executive Officers 
identified the most common performance indicators with the number of 
states that used them (Christkl 1997). 

Table 1: Reporting Indicators and States Adopting Them 



Indicators States 

Retention/ Graduation rates 32* 

Transfer rates 25* 

Faculty workload 24 

Satisfaction surveys 23 

Sponsored research funds 23 

Remediation activities/ effectiveness 21 

Pass rates on licensure exams 2 1 

Degrees awarded 20* 

Job Placements 19 

Admission standards 18* 



Number and percent of accredited programs 13 

Other common indicators 

Enrollment Trends* 

K-16 collaboration 

Racial diversity of students, faculty, and staff * 

Time-to-degree* 

Tuition and fees* 

(* Notes the indicators in the California Performance Report) 

The reports concentrate on undergraduate education, which 
constituted the major concern of external critics. Conventional and easy- 
to-capture input indicators, such as enrollment trends, high school 
averages, and standardized test scores of first-time students appear in 
most reports. Although they often include common process indicators, 
such as student/ faculty ratios and faculty workloads, most also contain 
more controversial and complex output and outcome measures. 
Performance reports present retention and graduation rates and time-to- 
degrees; results from satisfaction surveys of students and alumni; job 
placement statistics; program accreditation; and licensure test scores. 
Additionally, the growing interest in state priorities explains the inclusion 
of items related to economic development, such as the number and type 
of graduates and the size of sponsored research. Indicators on 
articulation between K-12 and higher education and on transfers 
between two- and four-year campuses suggest heightened concern with 
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the connection between all levels of education. 



Most of the reports present trends in institutional performance over 
time and often show comparisons with peer institutions, both in and out 
of state. Some Reports — such as California’s — present only data by 
institutional sectors rather than by individual campuses. A growing 
number of the reports allow institutions to select an indicator or two 
related to their specific missions. Despite these precautions, campus 
leaders fear that state officials and the popular media will make unfair 
comparisons and misuse the results by failing to recognize differences of 
institutional types and missions. 

Some critics from campus claim that each college or university is so 
different that it must have only unique indicators. This claim seems hard 
to sustain, since colleges and universities award the same degrees. Some 
academics also see external insistence on performance reporting as an 
invasion of institutional autonomy. They forget that the only institutions 
that are truly autonomous are those that fully self-supporting. No college 
or university - I know — deserves or desires that designation. 

By 2000, 30 states had performance reports. Although the spread of 
accountability reporting appears clear - the use of its results by state 
and campus policy makers remains cloudy (Burke, Rosin, Minassians, 
and Lessard 2000). An officer of the New Mexico Commission on Higher 
Education lamented in 1993: We... put together a report which we 
thought would provide more useful information. So far, we’ve had the 
same reaction to the latest report that we had to the first two reports, 
which was silence.” The Chronicle of Higher Education commented: “The 
lack of response makes some people wonder what’s the point of it all” 
(Mercer, 1993). 

A survey we did last year asked State Higher Education Finance 
Officers (SHEFOs) to estimate the extent that accountability reporting 
has improved the performance of public colleges and universities in their 
state. The replies reveal a limited impact on institutional performance in 
most states, with 49 percent saying little or no effect, and 38 percent 
claiming considerable or some effect. The rest could not assess the 
impact. The Finance Officer from California thought its report had little 
effect on performance. 
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Table 2: Im] 


pact Performance Reporting on Campus Performance 




Percent 

Responded 


States with Performance Reporting 


Great Extent 






Considerable 

Extent 


17.2% 


Kentucky, Louisiana, Missouri, Oregon, West Virginia 


Some Extent 


20.7% 


Illinois, Maryland, Mississippi, New Jersey, South 
Dakota, Wisconsin 


Little Extent 


37.9% 


Alabama, California, Florida, Hawaii, Idaho, 
Massachusetts, New Mexico, Rhode Island, South 
Carolina, Texas, Utah 


No Extent 


6.9% 


Arizona, Washington 


Cannot Assess 


17.2% 


Colorado, Connecticut, North Dakota, Ohio, Tennessee, 
Wyoming 



Our study of scores on the state-by-state report card, issued by the 
National Center for Public Policy and Higher Education, suggests that 
states with performance reporting received no better grades than those 
without this policy (2000). Many states with performance reports did 
poorly on the report cards, in part, because they did not contain critical 
indicators reflecting state needs, such as adult degree attainment, college 
going rates, high school performance, college cost as a percent of family 
income, and the state’s economic and civic benefits from higher 
education. But California, which includes those measures in its Report, 
did fairly well, with a B average. Of course, the existence or absence of 
performance reports did not cause the grades. Pat Callan, President of 
the National Center, claimed that differences depended mostly on 
“geography, income, wealth, and ethnicity (Selingo 2000). 

Table 3: California’s Grades 



Preparation 


C- 


Participation 


B+ 


Affordability 


A 


Completion 


C 


Benefits 


B+ 



The lack of fiscal consequences helps to explain the neglect of the 
performance reports by state and campus policy makers. Policies not 
connected to budgets get little attention in state capitals or on college 
campuses. The move from reporting to budgeting for performance 
seemed logical. It is hard to imagine that state and system officials 
would have reports on the accomplishments of colleges and universities 
and then totally ignore them when allocating scarce resources. 
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Progressing from reporting to funding seemed a short step to state policy 
makers but a momentous move to campus leaders. 

Traditional Budgeting 

States traditionally budgeted public colleges and universities based 
largely on current costs, student enrollments, and inflationary increases 
(McKeown 1996). These input, or resource factors, ignored the quantity 
and quality of graduates and the range and benefits of services to states 
and society. This cost-plus budgeting also promoted inappropriate 
growth in expenditures, enrollments, and programs, even in states with 
declining demographics and decreasing student demands. Some states in 
the 1980s did recognize institutional performance. They provided front- 
end funding to encourage desired campus activities in research and 
programs that encouraged economic development. Florida, New Jersey, 
and Ohio adopted this approach to create centers of excellence in their 
colleges and universities. They dropped nearly all of these initiatives 
during the recession and the reduction in state funding for higher 
education in the early 1990s. 

State funding for performance differs from these earlier efforts by 
allocating resources for achieved rather than promised results. This 
practice shifts somewhat the budget question from what states should do 
for their campuses toward what campuses do for their states and their 
students. The shift is slight in all states, since the sums allocated to 
performance remain relatively small, from less than one to about five 
percent of state operating support. Whatever the future of using campus 
performance in budgeting, the workload measures of current costs, 
student enrollments, and inflationary increases will — and should — 
receive the lion’s share of state allocations. The real issue is whether 
performance should count for something in state funding and whether it 
could make a difference. 

Performance Funding and Performance Budgeting 

Using performance in state allocations for public colleges and 
universities takes two different forms. The two approaches are 
sometimes confused, because of common characteristics, despite 
distinct differences (Burke and Serban 1998). [Slide 5] 



Performance funding ties tightly specific allocations to institutional 
results on each of the designated indicators. The tie is automatic and 
formulaic. If a campus achieves a set target on its graduation rate, it 
receives a specific amount of performance money. Performance 
funding focuses on budget distribution. 
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Performance budgeting allows governors and legislators or 
coordinating or system boards to consider campus performance on 
the indicators collectively as merely one factor in determining the 
total allocation for an institution. The link is loose and discretionary. 
This approach usually concentrates on budget preparation and 
presentation, and slights, or even ignores, budget distribution. 

The advantages and disadvantages of each of these approaches are the 
reverse of the other. In performance funding, the tie between results and 
resources is clear but inflexible. In performance budgeting, the link is 
flexible but unclear. 

The visible relation of resources to results in performance funding 
provides a tangible incentive for institutions to improve performance. On 
the other hand, this automatic link to funding may punish campuses in 
cases where performance falls because of circumstances beyond their 
control. The tight tie of funding to each indicator and the small sums 
usually allocated to performance funding also limit the number of 
indicators, since too many measures would trivialize important objectives 
with trifling sums. This restriction excludes many of the critical 
objectives of colleges and universities. 

In performance budgeting, the loose link of funding to results gives 
state governments or coordinating or system boards discretion over 
allocations based on performance, which allows consideration of campus 
circumstances. The absence of a direct tie of funding to each indicator 
also permits the use of a longer list of performance measures, which can 
include more of the many objectives of higher education. Using more 
measures in performance budgeting diminishes the impact of problems 
of validity and reliability with individual indicators, as opposed to 
performance funding where every measure counts for money. On the 
downside, the loose link of resources to results in performance budgeting 
reduces the incentive for institutions to improve performance. 

The growing popularity of linking state budgeting and funding to the 
performance of public campuses is unmistakable. Twenty-eight States 
(56%) now have performance budgeting; and 17 (34%], performance 
funding. Ten states have both programs, including California. Four years 
ago when our Surveys began, less than half of the states had one of these 
programs. Now, nearly three quarters of them link in some way state 
resources to campus results. 
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Table 6: States with Performance Budgeting 



Surveys 


Number (Percentage) 


States 


First 

April, 1997 


16 states (32%) 


Colorado, Florida, Georgia, Hawaii, Idaho, Illinois, 
Indiana, Iowa, Kansas, Mississippi, Nebraska, North 
Carolina, Oklahoma, Rhode Island, Texas, West 
Virginia 


Fourth 

June, 2000 


28 states (56%) 


Alabama, California, Connecticut, Florida, Georgia, 
Hawaii, Idaho, Illinois, Iowa, Kansas, Louisiana, 
Maine, Maryland, Massachusetts, Michigan, 
Mississippi, Missouri, Nebraska, Nevada, New Jersey, 
New Mexico, North Carolina, Oklahoma, Oregon, 
Texas, Utah, Virginia, Wisconsin 



Table 7: Performance Funding 1997-2000 



Surveys 


Number (Percentage) 


States 


First 

April, 1997 


10 states (20%) 


Colorado, Connecticut, Florida, Kentucky, Minnesota, 
Missouri, Ohio, South Carolina, Tennessee, 
Washington 


Fourth 

June, 2000 


17 states (34%) 


California*, Colorado, Connecticut, Florida, Illinois*, 
Kansas, Louisiana, Missouri, New Jersey, New 
York**, Ohio, Oklahoma, Pennsylvania, South 
Carolina, South Dakota, Tennessee, Texas 



* 2-year colleges only 

** State University of New York System 



Impact on Performance 

The bottom-line in assessing both performance funding and 
budgeting is the extent to which each has improved institutional 
performance. A realistic assessment is still premature, since nearly all of 
these programs are products of the mid to late 1990s, and most have 
been implemented for only a few years. Not surprisingly, nearly half of 
the Finance Officers in our 2000 survey say it is too early to evaluate the 
effect performance funding on institutional improvement, including the 
one from California. But 35 percent claim that the program has improved 
performance to a great or considerable extent. They cite great extent in 
South Carolina and Tennessee and considerable extent in Connecticut, 
Missouri, Ohio and Oklahoma. Respondents indicate some extent in 
South Dakota and little extent in Florida and Texas. None said no extent. 
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Table 8: To what extent I 


bas Performance Funding improved performance? 




Percent 


States with Performance Funding 


Great Extent 


11.8% 


South Carolina, Tennessee 


Considerable 

Extent 


23.5% 


Connecticut, Missouri, Ohio, Oklahoma 


Some Extent 


5.9% 


South Dakota 


Little Extent 


11.8% 


Florida, Texas 


No Extent 






Cannot Assess 


47.1% 


California, Colorado, Illinois, Kansas, Louisiana, New 
Jersey, New York, Pennsylvania 



Program duration and funding levels clearly affect these estimates. 
Tennessee, Missouri, South Carolina, and Ohio have had programs for 
some time and have supported them with considerable funding. Although 
Florida’s effort has existed for five years, its university sector has 
received scant funding in the last few budgets, although its two-year 
sector has a large funded program in workforce development. (Florida 
has now abandoned performance funding for its university sector.) Even 
respondents from states that give low ratings on the effect on 
improvement say that performance funding has caused campus leaders 
to concentrate more on institutional performance on the program 
objectives. 

Although performance budgeting has a somewhat longer history, still 
32 percent of the finance officers considered it too early to assess its 
impact. No program gets a rating of great extent. Respondents believe 
performance budgeting has improved campus performance to a 
considerable extent in 18 percent, and to some extent in seven percent of 
the programs. However, they think it has had little effect in 36 percent of 
the states, and no effect in 7 percent. All of the programs cited as having 
considerable or some effect on campus performance also had 
coordinating or system boards that consider performance in campus 
allocation. The effect on improved performance appears to depend on 
fiscal consequences, which is the rationale for both performance 
budgeting and performance funding. The loose link of performance to 
allocation in performance budgeting, as opposed to the tight tie in 
performance funding, seems to explain why the former appears to have a 
lesser impact on performance. 
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Table 9; To what extent has Performance Budgeting improved the performance? 





Percent 

Responded 


States with Performance Budgeting 


Great Extent 






Considerable 

Extent 


17.9% 


Missouri, Oklahoma, Oregon, Virginia, Wisconsin 


Some Extent 


7.1% 


Idaho, North Carolina 


Little Extent 


35.7% 


Hawaii, Massachusetts, Mississippi, Nebraska, Nevada, 
New Jersey, New Mexico, Texas, Utah 


No Extent 


7.1% 


Georgia, Kansas 


Cannot Assess 


32.1% 


Alabama, California, Connecticut, Florida, Illinois, Iowa, 
Maine, Maryland, Michigan 



The survey suggests that it is still too soon to assess their effect, 
given the short history of both programs. It does show that the 
performance funding has more effect than performance budgeting and 
that the impact of both approaches increases in relation to the clarity 
and the level of the fiscal consequences. On the other hand, too much 
funding can have the detrimental effect of producing budget instability. 
The early effort in South Carolina to base all funding on performance 
presents a classic example of this flaw. 

Five State Survey on Performance Funding 

Last year, we surveyed the presidents, vice presidents, academic 
deans, and department chairs from all two-and four -year public colleges 
and universities in Florida, Missouri, Ohio, South Carolina, and 
Tennessee. The replies from nearly 2000 campus leaders - a response 
rate of 45 percent - afford a closer look at the attitudes toward 
performance funding and its impact on public colleges and universities 
(Burke and Lessard, forthcoming). 

The results counter the common complaint in state capitals that 
campus leaders reject performance as a component in state budgeting for 
public colleges and universities. The Survey asked campus 
representatives to rate the “importance” of major factors in state 
budgeting for public higher education. Campus leaders consider all these 
factors important, but with decided preferences. They place inflation and 
salary increases first in importance, but see campus performance second 
and current cost third. Other factors receive less support, with neu; 
programs fourth, enrollment levels fifth, and special initiatives sixth. 
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Table 10; Budget Factors 





Two-Year 

Important 


Four-Year 

Important 


Inflationary &, Salary Increases 


87.5% 


87.9% 


Campus Performance 


81.7% 


83.0% 


Current Costs 


81.7% 


81.8% 


New Programs 


81.7% 


71.6% 


Enrollment Levels 


71.7% 


69.0% 


Special Initiatives 


68.4% 


66.0% 



Four-year respondents rate campus performance second, while it ties 
with current costs and new programs for that place in rankings by 
representatives of two-year institutions. Senior administrators favor new 
programs slightly more than campus performance, while chairs and 
especially deans see the latter as more important. Campus leaders 
clearly concede the importance of performance in state budgeting in 
theory, but — as their responses to later questions reveal — they quarrel 
with performance funding as practiced in their state. 

Indicator Preferences 

Overall, a majority of the respondents consider appropriate all of the 
indicators frequently found in performance funding programs, except 
time-to degree. The division between the indicators favored by campus 
leaders and those prized by government officials demonstrates the 
differences between academic concerns and capital desires. Some of the 
indicators thought least appropriate on campus are those most desired 
in state capitals. Governors and legislators push for undergraduate 
access, K-12 linkage, two to four year transfers, faculty workloads, new 
student preparations, standardized tests, and time to degree. 
Unfortunately, low rankings of indicators on diversity of students, faculty, 
and staff appeaur on campus as well as in state capitals. 
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Table 1 1 ; Performance Funding Indicators 



Ranked by Mean Scores 


Two-Year Rank 


Four-Year Rank 


1 . Accredited Programs 


3 


1 


2. Professional Licensure exams 


4 


3 


3. External Peer Review 


9 


2 


4. Employer Satisfaction Surveys 


1 


7 


5. Graduate Job Placement 


2 


6 


6. Alumni Satisfaction Survey 


7 


5 


7. Retention/ Graduation Rates 


8 


4 


8. Student Satisfaction Surveys 


5 


9 


9. Undergraduate Access 


6 


11 


10. Administrative Size/ Cost 


10 


8 


1 1 . Faculty Workload 


14 


10 


1 2. New Student Preparation 


12 


12 


13. K-12 Linkage 


11 


16 


1 4. Diversity of Students 


16 


13 


1 5. Standardized Test Scores 


15 


14 


1 6. Diversity of Faculty/ Staff 


17 


15 


1 7. Two to Four Year Transfers 


13 


18 


1 8. Time to Degree 


18 


17 



The ratings refute the common contention of a conflict on indicators 
between leaders of two and four-year campuses. Those from both sectors 
jointly favor nine of their top ten indicators. They also share low ratings 
of four measures: time to degree, diversity of faculty/ staff, standardized 
test scores, and - astonishingly for community colleges — two to four year 
transfers. Slightly less than half of the respondents from four-year 
institutions and just a bare majority of those from two-year colleges favor 
the transfer indicator. Although respondents from both sectors share 
more preferences than expected, those from four-year units show greater 
preferences for retention/ graduation rates and external peer reviews, 
while those from two-year colleges support more job placement, student 
access, and satisfaction surveys of employer and students. 

Campus Use of Performance Results 

The Survey asks respondents to assess the use of performance 
results for decisions in nine campus activities often targeted in 
performance funding. Overall, leaders cite moderate rather than 
extensive use in all areas, except faculty workload, which gets only 
minimal use. They claim institutional planning receives the most use, 
followed by student outcomes assessment, and — surprisingly — by 
internal budget allocation. Most experts think that that internal 
budgeting for schools and departments usually ignores performance in 
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favor of the traditional factors of student enrolments and faculty salaries. 
Authors of case studies for the five states in a book we are publishing on 
performance funding confirm this conclusion 

Averages for two-year colleges suggest significantly more use of 
performance results than those from four-year campuses. This result 
possibly reflects more administrative than faculty control of campus 
activities in community and technical colleges. Leaders of two-year 
institutions rate the use of performance results in institutional planning 
as extensive and even claim moderate use in faculty workload. 
Admissions is an expected exception. They place curriculum and planning 
second, student outcomes assessment third, and internal budget 
allocations fourth. Overall, senior campus officers perceive more use of 
performance results than deans and chairs. On the other hand, deans 
see less use than chairs in internal budgeting. 



Table 12: Use of Performance Results 





Two-Year Mean 
Scores 


Four-Year 
Mean Scores 


Academic Advising 


3.24 


3.66 


Administrative Services 


3.09 


3.54 


Admissions 


3.36 


3.28 


Curriculum 85 Planning 


2.77 


3.32 


Faculty Workload 


3.48 


3.58 


Institutional Planning 


2.46 


2.80 


Internal Budget Allocations 


2.95 


3.12 


Student Outcomes Assessment 


2.79 


3.13 


Student Services 


3.20 


3.55 



* Scale: 1 Very extensively, 2 Extensively, 3 Moderately, 4 Minimally, 5 Not at all 



Purposes of Performance Funding 

Contrary to popular impressions, campus leaders appear to accept all 
of the purposes of performance funding — including external 
accountability, but, again, they have decided preferences. They clearly 
favor institutional improvement, followed closely by increased state 
funding. Respondents see meeting state needs as trailing in third place 
and view external accountability as a distant fourth purpose. Campus 
leaders in all positions and both sectors endorse this overall order of 
preference. Most of the state replies repeat this ranking. However, those 
from South Carolina rate increased states funding even above 
institutional improvement; and Tennessee places external accountability 
before meeting state needs. 
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A follow up question asks whether the program in their state 
achieved these purposes. Only achievement of external accountability - 
the least preferred purpose - wins agreement from a majority of the total 
respondents. About a third believe that the program improved 
institutional performance and responsiveness to state needs, but just 15 
percent think that it increased state funding. The highest rates of 
agreement in all the states come on accountability, which campus 
leaders view as a less preferred purpose. On their favorite items, 
agreement is far less than desired on improving institutional 
performance, and nearly disappears on increasing state funding for 
public higher education. This pattern remains consistent for positions 
and sectors. 

The high value placed on increasing state funding and the low rating 
of its achievement may explain some of the negative assessments of 
performance funding. Actual state appropriations dashed the hope of 
increased funding in most of the study states (Palmer, Hines, and Reed 
1999). Only Florida (64%) and Missouri (45%) experienced increases in 
state appropriations over the last five years that exceed the national 
average of 32 percent. Ohio (31%) fell just short of that average. South 
Carolina averaged only 25 percent and Tennessee just 9 percent. Only 12 
percent of replies from Florida and 30 percent from Missouri agree that 
the program increased overall state funding. Reactions by states to the 
achievement of the goal of increased state funding suggests that 
performance funding got the blame for scant increases and no credit for 
improved appropriations. 

Impact on Campus Performance 

Respondents say that performance funding has only a minimal 
impact on most of the goals often designated in performance funding 
plans. They believe that performance funding has a moderate impact 
only on mission focus and administrative efficiency. Significantly, 
performance funding is viewed as having the most impact on objectives 
largely controlled by senior administrators and a minimal effect on 
outcomes depending on faculty activities. (Inter-institutional cooperation — 
which senior officers also control — did receive only a minimal rating, but 
this objective is generally not a favorite goal of top administrators.) 
Responses from both sectors and all positions give moderate rankings 
only to mission focus and administrative efficiency, except that business 
officers include only mission focus in that category. 

Although mission focus and administrative efficiency constitute 
important objectives, they hardly represent the only desired goals. State 
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officials and the general public are also concerned about faculty 
performance, faculty-student interaction, job placements, and the quantity 
and quality of student learning. 





Very 

High 


High 


Moderate 


Minimal 


No 

Effect 


Total 


Administrative 

Efficiency 


5 . 0 % 


15 . 3 % 


29 . 1 % 


28 . 4 % 


22 . 2 % 


100 % 


Faculty 

Performance 


3 . 2 % 


10 . 3 % 


29 . 5 % 


31 . 7 % 


25 . 2 % 


100 % 


Faculty -Student 
Interaction 


2 . 0 % 


8 . 1 % 


22 . 8 % 


30 . 9 % 


36 . 2 % 


100 % 


Graduate’ 

Further 

Education 


1 . 5 % 


5 . 7 % 


1 7 . 0 % 


28 . 8 % 


47 . 0 % 


100 % 


Graduates’ Job 
Placement 


2 . 7 % 


7 . 7 % 


16 . 9 % 


26 . 6 % 


46 . 1 % 


100 % 


Inter-Institutional 

Cooperation 


2 . 3 % 


8 . 0 % 


24 . 8 % 


31 . 4 % 


33 . 6 % 


100 % 


Mission 

Focus 


7 . 6 % 


21 . 0 % 


32 . 4 % 


20 . 5 % 


18 . 5 % 


100 % 


Quality & 
Quantity of 
Student Learning 


4 . 3 % 


12 . 8 % 


25 . 5 % 


29 . 3 % 


28 . 1 % 


100 % 


Research 

Funding 


3 . 7 % 


8 . 3 % 


18 . 7 % 


19 . 3 % 


49 . 9 % 


100 % 



Program Future 

Despite their critical critique of the use and impact of performance 
funding, respondents from all positions, both sectors, and every state 
consider performance funding as likely to continue. Seventy-one percent 
of the respondents from both two and four-year institutions say they 
consider continuation likely. Occupants of all positions overwhelmingly 
endorsed this prediction, although senior officers are more positive than 
deans and chairs. Clearly, the belief in continuation comes from support 
in state capitals rather than from campus preference, since campus 
leaders seem critical of the program and doubtful of its use in campus 
activities and its impact on institutional objectives. 
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Table 14: Program Future by the Type of Institution 





Likely 


Uncertain 


Unlikely 


Two-Year 


70.5% 


26.6% 


2.9% 


Four-Year 


71.1% 


27.1% 


1.7% 



Lack of Familiarity 

The failure of performance funding to have more impact on campuses 
may well result from a lack of familiarity with the program and its results 
by academic deans and department chairs — those most responsible for 
performance on campus. Survey respondents certainly had the 
opportunity to become familiar with the performance funding in their 
states. Nearly half held their positions for seven year or more, and two- 
thirds for at least four. Despite this longevity, they appear less familiar 
with performance funding than expected. Only half of the total 
respondents claim they are “very familiar” or “familiar” with the program 
in their state, although overall those from two-year campuses seem more 
aware than their four-year counterparts. Lack of familiarity appears even 
in South Carolina with its controversial program and in Tennessee, 
which started performance funding officially the late 1970s. Controversy 
and longevity clearly affected familiarity but not nearly as much as 
anticipated. State responses vary widely on familiarity, from highest in 
South Carolina to the lowest in Ohio. 



Table 15: Familiarity with Performance Funding by State 
Very Familiar/Familiar 
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Table 16: Familiarity With Performance Funding 

Very Familiar Somewhat Slightly Not At All 
Familiar Familiar Familiar Familiar 

Two-Year 23.8% 33.8% 25.9% 11.3% 5.2% 100.0% 

Four-Year 16.5% 29.0% 28.5% 17.4% 8.6% 100.0% 

Performance funding becomes increasingly invisible on campus below 
presidents and vice presidents. Nearly 90 percent of the senior officers 
say they are familiar with the program, but over 40 percent of the deans 
and over 60 percent of the chairs admit little or no familiarity. Even in 
South Carolina, more than a quarter of the deans and nearly 40 percent 
of the chairs say they are only “somewhat,” “slightly,” or “not at all” 
familiar. Despite the longevity of the program in Tennessee, over a third 
of deans and more than half of the chairs give the same reply. Senior 
officers in two-year institutions are less familiar and deans and chairs 
more familiar with performance funding than their four-year 
counterparts. 



Table 17: Familiarity by Sector and Position 



Position 




Very 


Familiar Somewhat 


Slightly 


Not At All 




Senior 


Two -Year 


Familiar 

49.3% 


38.2% 


Familiar 

10.7% 


Familiar 

1.8% 


Familiar 


100.0% 


Admin 


Four-Y ear 


62.2% 


27.9% 


6.3% 


2.7% 


.9% 


100.0% 


Academic Two-Y ear 


22.5% 


41.7% 


30.0% 


5.0% 


.8% 


100.0% 


Deans 


Four-Y ear 


20.9% 


33.3% 


25.3% 


15.1% 


5.3% 


100.0% 


Depart 


Two-Year 


10.1% 


29.0% 


33.2% 


18.4% 


9.3% 


100.0% 


Chairs 


Four- Year 


8.8% 


27.9% 


32.6% 


20.1% 


10.6% 


100.0% 



Why Haven’t Performance Programs Made More DifTerence 

The lack of familiarity of deans and chairs with performance funding 
may well explain why all of the performance programs have not made 
more of a difference. Policies on reporting, budgeting, and funding hold 
coordinating boards accountable for the collective performance of higher 
education in a state, and presidents and vice presidents responsible for 
institutional performance. Higher education has decentralized authority 
over most academic matters to divisions and departments. But few 
colleges or universities decentralize accountability to their operating 
units, which are really responsible for the results on most of the 
indicators in these performance programs. How many department chairs 
on your campus can tell you the graduation rates or time to degree of 
their majors or even the tuition or fees for their students? 
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The lack of familiarity with performance results flows from the 
absence of wide dissemination of the performance of internal units and of 
the lack of fiscal consequences for divisions and departments. To remedy 
these problems, institutional research should provide deans, chairs, and 
senior officer on campus with annual reports of the performance of 
divisions and departments on the indicators used in reporting, 
budgeting, and funding programs. Statewide programs in budgeting and 
funding also could encourage internal concern with institutional 
performance by adding an indicator on internal campus allocations. They 
should provide special funding for colleges or universities that use 
appropriate statewide indicators in their internal budgeting of divisions 
and departments. Such a step would leverage state funding with 
institutional allocations. More important, this provision would engage 
directly the departments and the professionals most responsible for 
performance in meeting the institutional goals set in performance 
programs. Professors are unlikely to pay attention to performance 
programs that neither publicize nor reward their own performance. 

Some of the most common performance indicators in state programs 
seem suited for inclusion in departmental allocations. After all, the 
actions of academic departments are most responsible for the results on 
the common indicators of retention, graduation, and transfer rates; 
licensure test scores; and the satisfaction levels of students and alumni. 
Such a proposal would certainly make performance funding more visible 
on campus. It would also clarify the link between planning, budgeting, 
and performance. 



Conclusion 

Our surveys and studies suggest that to date reporting, budgeting, 
and funding performance have made some difference, but not nearly as 
much as their advocates promised. Opponents seize on the limited 
results as a reason for abandoning all efforts at holding public colleges 
and universities accountable for their results. They exhibit the common 
fault on campus of condemning new proposals for institutional change 
without examining the flaws in current practices. A classic case of this 
double standard occurred a year ago, after 1 gave a paper on performance 
funding at a national conference. A president of a leading university 
demanded: “what has this program done to improve the performance of 
colleges and universities?” Instead, of responding with the obvious 
answer that the policy was too recent to assess the impact, 1 asked, 
“what has traditional budgeting after thirty years done to improve 
campus performance?” After several moments of embarrassed silence. 
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the President blurted: “Why traditional budgeting was never intended to 
improve performance.” 1 smiled and said: “1 rest my case.” 

It is still too soon to tell whether performance programs will 
eventually improve the results of colleges and universities, or increase 
their responsiveness to state and student needs. But one conclusion is 
clear. Higher education is too important and too costly to states and 
students to consider only resource inputs and ignore performance 
results. We academics are too good at criticizing the outcomes of outside 
organizations to plead the impossibility of evaluating our own 
performance. We may be able to persuade governors and legislators that 
only educators can evaluate the performance of higher education, but 
they will never accept the answer that it cannot be done. Taxpayers are 
unlikely to accept forever the proposition that performance should count 
in all endeavors except higher education. The real question is whether 
educators and experts in institutional research will lead or leave the 
action on accountability for results to outsiders. Community colleges 
should champion the new concept of accountability and educational 
excellence for your claim to fame is based on results not resources, on 
performance not prestige. 
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